Embedded Systems

A Comprehensive Safety Metric to Evaluate Perception in Autonomous Systems

by Georg Volk, Jörg Gamerdinger, Alexan­der von Bernuth, and Oliver Bring­mann
In 2020 IEEE 23rd In­ter­na­tional Con­fer­ence on In­tel­li­gent Trans­porta­tion Sys­tems (ITSC), pages 1-8, 2020.

Key­words: Mea­sure­ment, Safety, Ob­ject de­tec­tion, Au­tonomous ve­hi­cles, Roads, Real-time sys­tems, Bench­mark test­ing

Ab­stract

Com­plete per­cep­tion of the en­vi­ron­ment and its cor­rect in­ter­pre­ta­tion is cru­cial for au­tonomous ve­hi­cles. Ob­ject per­cep­tion is the main com­po­nent of au­to­mo­tive sur­round sens­ing. Var­i­ous met­rics al­ready exist for the eval­u­a­tion of ob­ject per­cep­tion. How­ever, ob­jects can be of dif­fer­ent im­por­tance de­pend­ing on their ve­loc­ity, ori­en­ta­tion, dis­tance, size, or the po­ten­tial dam­age that could be caused by a col­li­sion due to a missed de­tec­tion. Thus, these ad­di­tional pa­ra­me­ters have to be con­sid­ered for safety eval­u­a­tion. We pro­pose a new safety met­ric that in­cor­po­rates all these pa­ra­me­ters and re­turns a sin­gle eas­ily in­ter­pretable safety as­sess­ment score for ob­ject per­cep­tion. This new met­ric is eval­u­ated with both real world and vir­tual data sets and com­pared to state of the art met­rics.